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•'v  by  pitch  by  roll.  The  results  did  indicate  that  use  of  the  ADI  ball  resulted  in  faster  recovery  times.  Significant 
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INTRODUCTION 

Maintaining  situational  awareness  is  a  major  task  in  piloting  an  aircraft.  Situational  awareness 
is  a  multi-faceted  phenomenon.  In  order  to  maintain  situational  awareness  a  pilot  needs  to  know 
the  relationship  of  the  aircraft  relative  to:  ( 1 )  the  surface  of  the  earth  or  other  surface  (e.g.r  an  air¬ 
craft  carrier),  (2)  geographical  coordinates,  and  (3)  other  aircraft  and  threats. 

Spatial  orientation  is  a  major  aspect  of  situational  awareness.  To  maintain  spatial  orientation  a 
pilot  needs  information  about  altitude,  airspeed,  and  attitude.  This  information  is  normally  availa¬ 
ble  via  several  sensory  channels.  The  pilot's  visual,  auditory,  vestibular,  and  kinestic  senses  all  con¬ 
tribute  information  about  his  orientation  in  space.  For  example,  the  pilot  receives  aural  cues  from 
the  radio  altimeter,  engines,  and  environmental  control  system.  Force  feedback  from  the  stick  and 
rudder  and  g-force  vectors  provide  kinestic  and  vestibular  cues.  In  addition,  it  has  been  estimated 
that  over  80  percent  of  all  information  is  processed  visually  (Senders,  1983).  Thus,  visual  cues  are 
the  primary  source  of  information  for  maintaining  spatial  orientation.  The  sources  of  a  pilot's  vis¬ 
ual  cues  are  the  out-of-the-cockpit  visual  scene  and  his  various  cockpit  attitude  displays. 

When  loss  of  situational  awareness  occurs,  the  pilot,  the  aircraft,  and  the  mission  are  in  jeopar¬ 
dy.  Loss  of  situational  awareness  has  been  identified  as  a  contributing  factor  in  several  aircraft  acci¬ 
dents  and  numerous  aircraft  incidents.  Four  F/A-18  and  eleven  F-16  accidents  were  attributed,  in 
part,  to  loss  of  situational  awareness.  Maintaining  situational  awareness  is  an  especially  difficult 
task  for  the  pilot  of  an  F/A-18.  Several  of  the  sources  of  information  concerning  situational  aware¬ 
ness,  previously  available  to  pilots  of  other  aircraft,  are  no  longer  available  or  have  been  reduced  in 
effectiveness  for  the  F/A-18  pilot.  The  introduction  of  fly-by-wire  technology  has  virtually  elimina¬ 
ted  the  force  feedback  from  the  control  stick  and  rudder  pedals.  Soundproofing  of  the  cockpit  and 
the  use  of  quieter  engines  have  reduced  the  effectiveness  of  aural  cues.  Thus,  when  flying  under 
IF  R  conditions,  during  which  there  is  no  out-of-the  cockpit  visual  scene,  the  F/A-18  pilot  must  rely 
on  the  primary  attitude  reference  indicator  (ARI)  as  the  major  source  of  information. 

The  primary  ARI  in  the  F/A-18  is  the  pitch  ladder  and  roll  scale  presented  on  the  head-up  dis¬ 
play  (HUD).  However,  in  response  to  a  questionnaire  developed  at  the  Naval  Air  Development  Cen¬ 
ter  (September,  1984),  pilots  rated  the  HUD  as  "marginally  acceptable"  for  recovery  from  unusual 
attitudes.  Several  aspects  of  the  HUD  format  may  have  contributed  to  this  less  than  satisfactory 
rating.  (1 )  Airspeed  and  altitude  information  are  presented  in  a  digital  format.  Pilot  comments  on 
this  method  of  presentation  have  generally  been  unfavorable.  (2)  The  digital  format  does  not  pro¬ 
vide  the  necessary  trend  information  that  is  required  in  extreme  environments.  (3)  The  digital 
readout  is  updated  so  quickly  during  rapid  maneuvering  that  it  is  difficult  to  read  the  display.  (4) 
Pitch  and  roll  scale  formats  were  also  identified  as  marginal  to  assess  attitude  quickly.  The  fourth 
problem  was  the  focus  of  this  research  effort. 

Taylor  (1984)  discusses  various  HUD  pitch  ladder  formats  in  terms  of  the  cognitive  processes 
involved  in  assessing  attitude.  Pitch  and  roll  indications  on  various  HUD  formats  are  complex  multi¬ 
dimensional  stimuli.  He  discusses  the  contribution  of  global  and  local  cognitive  processing  and  re¬ 
dundancy  to  the  processing  of  this  information  for  assessment  of  attitude. 

Results  from  Taylor's  research  suggest  several  dimensions  of  the  pitch  ladder  formats  which 
might  be  modified  to  improve  these  formats  as  attitude  indicators.  Based  on  Taylor's  results, 
several  modifications  to  the  current  pitch  ladder  presented  on  the  F/A-18  HUD  were  made  by 
NADC.  An  experimental  paradigm  was  developed  to  test  the  current  and  the  modified  pitch  ladder 
along  with  a  typical  ADI  ball. 
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The  ADI  ball  was  included  in  the  experiment  as  a  result  of  discussions  with  a  group  of  pilots, 
psychologists,  and  engineers  at  the  Naval  Air  Test  Center  (April,  1 985).  It  was  evident  that  pilots,  if  given  a 
choice,  prefer  to  have  an  ADI  ball  present  as  an  attitude  indicator.  Although  there  is  an  ADI  ball  located  in  the 
instrument  suite  of  the  F/A-1 8,  it  was  considered  too  small  and  poorly  located  to  be  useful  to  the  pilots  as  a 
secondary  instrument. 

The  suggestion  resulting  from  thediscussion  at  NATC  was  to  optimize  the  location  of  the  ADI 
ball  and  include  it  as  a  cross  reference  for  attitude  indication  along  with  the  pitch  ladder  on  the 
HUD.  Roscoe,  Corl,  and  Jensen  (1981)  discuss  several  features  of  an  ADI  ball  which  suggest  that  it 
would  be  superior  to  a  pitch  ladder  for  attitude  indication.  The  purpose  of  the  current  research  ef¬ 
fort  was  to  compare  the  merits  of  two  HUD  pitch  ladders  and  an  ADI  ball  in  recovering  from  unusu¬ 
al  attitudes. 

Situational  awareness  and  particularly  spatial  orientation  are  compromised  when  conflicting  in¬ 
formation  is  received  from  the  various  sources  of  attitude  information.  A  working  group  at  NATC 
identified  several  situations  during  which  loss  of  situational  awareness  was  most  likely  to  occur.  In 
tactical  situations  it  was  most  likely  to  occur  during  air  combat  maneuvering  (ACM),  all  weather  in¬ 
strument  flight  (AWI),  and  during  bombing  runs  when  target  fixation  might  occur.  In  non-tactical 
environments  loss  of  situational  awareness  was  most  likely  to  occur  during  takeoff  and  catapult 
launch  and  on  instrument  landings.  The  non-tactical  environments  in  which  situational  awareness 
was  most  likely  to  be  lost  usually  involved  flying  against  a  homogeneous  background  (i.e.,  flying  in 
clouds,  clear  blue  sky  over  water,  at  night,  or  against  solid-colored  earth).  The  lack  of  a  real,  visible, 
horizon  is  one  of  the  primary  causes  of  loss  of  situational  awareness. 
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METHODOLOGY 

The  current  research  effort  was  divided  into  two  experiments.  The  first  experiment  was  a  static 
comparison  of  two  pitch  ladder  formats  and  a  pictorial  representation  of  a  typical  ADI  ball.  The  ability  of 
these  formats  to  aid  the  subject  in  deciding  how  to  recover  from  unusual  attitudes  was  assessed.  The 
hypotheses  for  this  experiment  were  that  the  ADI  ball  would  result  in  the  fastest  decision  times  and  the 
fewest  errors,  followed  by  the  revised  pitch  ladder,  with  the  use  of  the  current  pitch  ladder  resulting  in  the 
slowest  decision  times  and  the  most  errors.  Decision  time  was  defined  as  the  amount  of  time  required  to 
assess  the  orientation  and  initiate  an  appropriate  control  response.  Errors  were  defined  as  inappropriate 
control  response,  given  a  particular  display  orientation. 


The  hypotheses  for  this  experiment  were  based  on  research  conducted  by  Roscoe  (1968)  and  by  Taylor 
(1984).  Roscoe’s  contention  was  that  a  display  which  more  closely  approximated  the  real  world,  (i.e., 
represented  a  contact  analog)  would  be  more  efficient  for  its  intended  purpose.  Therefore,  given  that  the  ADI 
represents  a  contact  analog  of  real  horizon  more  so  than  a  pitch  ladder,  it  is  logical  to  assume  that  the  ADI 
would  be  superior  to  the  pitch  ladder  as  an  attitude  indicator.  The  revised  pitch  ladder  incorporated  several 
features  which  resulted  in  faster  reaction  times  in  Taylor's  experiments.  Thus,  it  was  hypothesized  that  the 
revised  pitch  ladder  would  be  superior  to  the  current  pitch  ladder. 


The  second  experiment  was  based  on  results  from  the  static  experiment.  The  three  display  for¬ 
mats  from  the  previous  experiment  were  tested  dynamically  in  a  medium-fidelity,  ground-based, 
simulator  in  the  Crewstation  Evaluation  Facility  (CREST)  at  NADC.  However,  due  to  software  lim¬ 
itations  which  did  not  allow  for  an  exact  duplication  of  the  revised  pitch  ladder  format  as  it  ap¬ 
peared  in  the  static  experiment,  only  the  current  F/A-18  pitch  ladder  and  an  ADI  ball  were  experi¬ 
mentally  compared.  The  modified  pitch  ladder  in  its  current  form  was  included  as  a  matter  of  inter¬ 
est,  but  was  not  included  in  the  formal  analysis. 

Two  experimental  hypotheses  were  proposed  for  the  dynamic  experiment.  The  first  hypothesis  was  that 
decision  times  would  be  faster  when  using  the  ADI  ball  than  when  using  the  current  pitch  ladder.  The 
second  hypothesis  was  that  recovery  times  defined  as  the  amount  of  time  between  the  initial  control 
response  and  when  the  subject  had  the  simulation  stable  at  a  straight  and  level  attitude,  would  be  faster 
when  using  the  ADI  ball  than  when  using  the  current  pitch  ladder. 
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EXPERIMENT  ONE:  STATIC  EVALUATION 


SUBJECTS 

Four  female  and  eight  male  subjects  were  employed  for  this  experiment.  None  of  the  twelve 
subjects  were  pilots.  Learning  effects  were  controlled  by  providing  a  practice  session  for  every  sub¬ 
ject  to  reach  a  90  percent  correct  response  criteria.  This  criteria  was  achieved  when  the  subject  suc¬ 
cessfully  corrected  17  out  of  18  attitude  disorientations  back  to  straight  and  level. 

APPARATUS 

The  experimental  apparatus  is  presented  in  F  igure  1 .  The  apparatus  consisted  of ;  ( 1 )  an 
Apple  II  plus  micro-computer,  (2)  a  Carroll  127  X  127  dot  matrix  (17  in.  X  17  in.)  rear  projec¬ 
tion,  touch  panel  screen,  (3)  two  Kodak  ectographic  B-2,  random  access,  slide  projectors,  and 
(4)  the  Workload  Assessment  Device  (WAD). 

The  WAD  consists  of  several  components  developed  by  Systems  Research  Laboratories  (SRL) 
for  NADC.  The  software  development  system  consists  of  a  micro  processor,  a  C/PM  operating  sys¬ 
tem,  three  disk  drives,  and  a  Micro-Term  ERGO  301 FK  monitor.  The  air  borne  unit  consists  of  a 
microprocessor,  a  Princeton  Graphics  Systems  SR-12  monitor,  and  a  Measurements  Systems, 
two-axis,  joystick  controller. 

The  complete  system  is  called  the  Design  By  Experimentation  system  (DBE).  A  systems  de¬ 
scription  is  available  in  the  users  and  systems  documentation  (Analytics,  1984).  The  system  was 
interfaced  to  an  EPSON  FX  80  dot  matrix  printer.  The  printer  was  used  to  obtain  hard-copy  out¬ 
put  from  the  system  for  data  reduction  and  analysis. 

STIMULI 

The  stimuli  for  this  experiment  were  three  attitude-indicator  formats.  The  three  formats  are 
represented  in  Figure  2.  Figure  2a  presents  the  pitch-ladder  currently  available  on  the  F/A-18 
HUD.  Figure  2b  presents  the  modified  pitch-ladder  developed  for  this  experiment.  Figure  2c  is 
a  pictorial  representation  of  a  typical  ADI  ball.  The  features  by  which  attitude  was  indicated  by 
each  of  the  formats  were  identified  and  explained  to  the  subjects.  The  pitch-ladder  formats  present 
pitch  information  by  the  relationship  of  the  velocity  vector,  ,  to  the  pitch  scale.  The  longer  of 
the  three  lines  represents  the  horizon.  The  shorter  lines  above  and  below  the  horizon  line  represent 
five  degrees  of  pitch  'ip  and  down,  respectively.  The  pitch  scale  goes  up  to  90  degrees  of  pitch  both 
up  and  down.  However,  only  three  lines  were  visible  at  one  time.  Roll  is  indicated  by  the  rotation 
of  the  entire  pitch  ladder  on  the  roll  axis,  about  the  velocity  vector. 

Both  pitch-ladder  displays  share  several  features.  They  are  both  outside-in  displays.  The  horizon  line  on 
the  pitch  ladder  moves  in  relation  to  movement  of  an  actual  horizon.  Thus,  if  the  aircraft  were  rolled  25 
degrees  to  the  right,  the  horizon  line  would  be  displayed  as  angled  25  degrees  to  the  left.  Both  pitch-ladder 
formats  also  have  solid  pitch  lines  when  pitched  up  and  dashed  pitch  lines  when  pitched  down.  The  pitch 
lines  also  increase  in  angle  with  increases  in  the  pitch  angle  of  the  aircraft  in  a  ratio  of  1 :2.  Therefore,  if  the 
aircraft  wer  pitched  40  degrees  the  pitch  line  would  be  angled  20  degrees.  The  slope  of  the  pitch  lines  is 
always  toward  the  horizon  for  both  pitch-ladders.  The  pitch  lines  for  both  pitch-ladders  have  horizon  markers 
which  point  to  the  horizon. 
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The  iwo  pitch-ladders  had  several  diffemt  features  which  comprised  the  experimental  manipulation.  The 
labels  for  each  pitch  line  are  on  both  sides  of  the  line  for  the  current  F/A-1 8  format  and  on  one  side  only  for 
the  revised  format  The  labels  are  beside  the  pitch  lines  for  the  current  format.  They  are  on  top  of  the  pitch 
lines  in  the  revised  format  but  only  when  upright  The  labels  remain  upright  even  when  rolled  180  degrees 
on  the  current  format  The  labels  rotate  relative  to  the  pitch  lines  on  the  revised  format.  The  labels  have 
negative  signs  to  indicate  pitch  down  on  the  revised  format  but  do  not  on  the  current  format.  The  horizon 
markers  on  each  of  the  pitch  lines  are  located  on  the  outside  of  the  lines  on  the  current  format  and  on  the 
inside  on  the  revised  format 

The  changes  mentioned  above  were  made  to  provide  more  redundant  cues  to  attitude  orienta¬ 
tion.  Tay'or  (1984)  suggested  that  these  changes  would  provide  stimuli  which  when  processed 
would  provide  globally-processed  cues  to  attitude.  Viewing  the  labels  on  the  right  of  the  pitch  lad¬ 
der  was  hypothesized  to  provide  a  global  cue  to  being  upright,  whereas,  if  the  labels  were  on  the  left 
they  would  provide  global  cues  that  the  aircraft  was  inverted.  If  the  labels  appeared  under  the  pitch 
lines  they  were  also  cues  to  being  inverted.  If  the  numbers  were  upside-down  they,  too,  were  cues  for 
being  inverted.  It  was,  therefore,  hypothesized  that  the  available  redundant  cues  of  the  revised 
pitch-ladder  would  be  a  better  indication  of  attitude. 

The  pictorial  representation  of  the  ADI  ball  had  several  features  which  indicated  attitude.  The 
ball  had  two  colors,  light-blue  and  black.  The  light  blue  area  of  the  ball  represented  the  sky  (i.e., 
pitch  up)  and  the  black  area  of  the  ball  represented  the  earth  (i.e.,  pitch  down).  The  area  where  the 
two  colors  met  was  the  horizon.  There  were  scale  markings  on  the  ball  which  indicated  five  degree 
changes  in  pitch.  There  was  also  a  roil  scale  located  around  the  ball.  It  was  a  fixed  scale  with  a 
moving  indicator.  There  were  three  cues  present  to  indicate  pitch.  The  location  of  the  horizon  rela¬ 
tive  to  the  housing  of  the  ball,  the  amount  of  one  color  versus  the  other,  and  the  reading  from  the 
pitch  scale,  provided  redundant  cues  to  pitch  orientation.  If  pitched  up  the  majority  of  the  ball  was 
light  blue  and  the  horizon  line  appeared  toward  the  top  of  the  ball  (unless  inverted).  If  pitched 
down,  the  majority  of  the  ball  was  black  and  the  horizon  line  appeared  toward  the  bottom  of  the 
ball.  There  were  two  cues  present  which  indicated  orientation  of  the  horizon  line  and  the  location 
of  the  indicator  on  the  roll  scale. 

The  stimuli  were  first  drawn  on  mylar  with  a  Versatic  Pen  Plotter  and  were  then  photographed 
on  35mm.  slide  transparency  film.  Fifty-four  slides  were  produced.  There  were  three  values  of 
pitch  employed  for  this  experiment:  0°,  55°,  and -55°.  Six  values  of  roll  were  used:  0°,  60°,  120°, 
180°,  -60°,  and  -120°.  Slides  were  produced  for  each  of  the  eighteen  possible  combinations  of 
pitch  and  roll,  for  alt  three  of  the  formats,  for  a  total  of  54  slides. 

PROCEDURE 

The  experimental  design  was  a  completely  within-subjects  design.  Each  subject  was  presented 
with  18  slides  for  each  of  the  three  display  formats.  The  order  of  display  format  and  pitch/roll 
combination  was  counter-balanced  to  control  for  any  potential  carry-over  effects. 

The  subjects  were  tested  in  the  Man-Machine  Interface  Laboratory  (MMIL),  within  the  CREST 
facility.  The  subjects  were  given  several  practice  trials  in  order  to  familiarize  them  with  the  stimuli 
and  the  experimental  procedure.  The  practice  trials  were  continued  until  the  subjects  reached  a  per¬ 
formance  criteria  of  90  percent  correct  responses.  The  experimental  trials  were  then  initiated. 
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Each  subject  received  54  trials.  Each  slide  was  a  unique  combination  of  one  of  three  levels  of 
pitch  and  six  levels  of  roll.  The  subjects  received  all  of  the  possible  pitch/roll  combinations  for  one 
format  before  proceeding  to  the  next  format.  A  trial  began  by  prompting  the  subject  that  a  slide 
was  about  to  be  presented.  The  experimenter  then  initiated  the  first  slide.  The  subject  viewed  the 
slide,  determined  the  orientation  presented  on  it,  and  made,  what  they  considered  to  be,  an  appro¬ 
priate  control  input  with  the  joystick.  The  subject  then  touched  the  screen  and  a  new  slide  was 
presented. 

DEPENDENT  MEASURES 

Two  measures  of  performance  were  collected.  The  first  measure,  decision  time,  was  defined  as 
the  amount  of  time  that  elapsed  between  the  onset  of  the  slide  and  the  first  movement  of  the  joy¬ 
stick.  The  joystick  position  was  sampled  every  0.1  seconds  and  the  first  deviation  from  0°  pitch 
and  0°  roll  was  considered  a  control  input.  The  second  measure,  error,  was  defined  as  a  joystick 
movement  in  an  inappropriate  direction  to  correct  the  displayed  orientation  back  to  straight  and 
level  (i.e.,  0°  pitch  and  0°  roll).  Errors  were  converted  to  a  percentage. 

RESULTS 

The  means  for  decision  time  and  the  percentage  of  error  for  each  of  the  display  formats  are 
presented  in  Table  1.  The  results  of  the  Friedman  test  for  matched  groups  indicated  a  significant 
difference  for  decision  time  (X2  ■  14.86,  p  <  0.001)  (Hays,  1973).  ANewman-Keuls  post-hoc  test 
(Hays,  1973)  revealed  that  decision  times  for  the  ADI  ball  were  significantly  faster  than  either 
pitch-ladder  format.  The  decision  times  for  the  two  pitch-ladder  formats  were  not  significantly  dif¬ 
ferent.  No  differences  for  percentage  of  error  were  revealed. 
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TABLE  1 

MEAN  DECISION  TIME  AND  PERCENTAGE  ERRORS 


DEPENDENT 

MEASURES 

DISPLAY  FORMATS 

F/A-18  HUD 

ADI  BALL 

PROPOSED  HUD 

MEAN  S.D. 

Dra 

MEAN  S.D. 

DECISION 

TIME 

4.14  1.03 

3.37*  0.94 

4.43  0.99 

PERCENTAGE 

ERRORS 

0.06  0.06 

0.09  0.11 

0.10  0.09 

•Significantly  different  (p  <  0.001 )  from  other  two  formats. 
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EXPERIMENT  TWO:  DYNAMIC  EVALUATION 


Experiment  two  dynamically  assessed  the  same  display  formats  used  in  experiment  one  with 
one  exception;  the  revised  pitch-ladder  was  not  included  in  the  formal  analysis.  The  hypotheses  for 
this  experiment  were:'  (1)  decisions  times  would  be  faster  for  the  ADI  ball  than  the  current  pitch- 
ladder  and  (2)  recovery  time9  would  be  faster  for  the  ADI  ball  than  the  current  pitch-iadder. 

The  experimental  design  included  only  the  current  F/A-18  pitch-ladder  and  an  ADI  ball.  The 
modified  format  could  not  be  reproduced  in  the  dynamic  simulator  exactly  a*  it  appeared  in  experi* 
ment  onen  Due  to  this  limitation  and  the  lack  of  significant  differences  between  the  two  pitch- 
ladder  formats  in  experiment  one,  the  modified  pitch-ladder  was  not  included  as  pari  of  the  formal 
experiment. 

The  only  difference  between  the  modified  pitch-ladder  format  from  experiment  one  and  the 
one  generated  by  the  simulator  were  the  labels  on  the  pitch  lines.  They  did  not  rotate  in  conjunc¬ 
tion  with  the  rotation  of  the  pitch  lines.  The  subjects  were,  however,  presented  with  the  modified 
format  after  they  completed  the  necessary  trials  for  the  current  pitch  ladder  and  ADI  formats.  The 
modified  format  was  not  subjected  io  formal  statistical  analysis. 

SUBJECTS 

Eight  non-pilot,  thre«>  female  and  five  male,  and  three  pilot  subjects  were  tested  for  this  ex¬ 
periment.  To  control  for  learning  effects  and  the  potential  for  bias  due  to  differential  exposure  to 
attitude  indicator  formats  by  the  pilots,  all  of  the  subjects  were  given  sufficient  practice  on  the 
simulator.  They  were  also  given  several  practice  trials  to  determine  baseline  performance  and  to 
familiarize  them  with  the  experimental  procedure.  The  subjects  were  required  to  meet  a  90  percent 
correct  performance  criteria  before  experimental  trials  were  initiated. 

APPARATUS 

The  system  used  in  this  experiment  was  the  dynamic  cockpit  simulator  in  the  CREST  facility  at  NADC. 
This  systam  is  presented  in  Figure  3.  The  system  consisted  of  a  Digital  Equipment  Corporation  PD P  1 1/34 
mini-computer  w*th  a  digital  to  analog  converter,  an  Evans  and  Sutherland  PS-300  symbol  generator,  a  Data 
General  NOVA-820  mini-computer  with  an  analog  to  digital  converter,  a  diffractive  optics  head-up  display 
(HUD),  an  ADI  bail,  a  large  screen  virtual  image  television  screen,  a  Sony  V-Matic  video  cassette  recorder,  a 
Bowmar  Programmable  Control  Panel,  and  a  U.S  Army  Type  C-1  control  stick. 


The  PDP-1 1/34  contained  an  aerodynamic  model  of  the  F/A-18  and  was  the  main  controller 
of  the  system.  It  also  sampled  the  pitch  and  roll  coordinates  every  0.1  seconds.  These  samples  were 
stored  In  memory  and  used  for  data  analysis.  The  Evans  and  Sutherland  symbol  generator  con¬ 
trolled  the  symbology  which  was  displayed  on  a  HUD,  The  outside  scene  which  provided  the  back¬ 
ground  against  which  the  subjects  flew  the  simulation  was  recorded  on  video  tape  and  projected  on 
the  large  screen  virtual  image  television  system.  The  NOVA  computer  controlled  the  dynamics  of 
the  ADI  ball.  More  exact  specifications  of  the  system  are  available  upon  request. 

STIMULI 

The  stimuli  for  this  experiment  were  actual  display  formats.  The  current  F/A-18  pitch-ladder 
was  presented  on  a  HUD,  The  HUD  was  developed  by  Hughes  Aircraft  Company  for  NADC.  It 
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has  a  diffractive  optics  combiner  with  a  field  of  view  of  20°  X  30°.  The  pitch-ladder  was  presented 
along  with  a  roll  scale,  a  heading  indicator,  a  digital  airspeed  indicator,  and  a  digital  altitude  indica¬ 
tor.  The  symbology  presented  on  the  HUD  was  identical  to  that  which  was  displayed  in  the  most 
decluttered  mode  of  the  F/A-18.  The  pitch-ladder  format  was  identical  to  the  format  described 
above  for  experiment  one. 

The  ADI  ball  used  in  this  experiment  was  the  actual  display  from  an  F-14.  An  F/A-18  ball  was 
not  available  for  the  experiment,  but  based  on  its  similar  characteristics  the  F-14  ball  was  selected. 

It  was  two  inches  in  diameter.  The  pitched-up  half  of  the  ball  was  light  gray  and  the  pitched-down 
half  of  the  ball  was  black. 

The  revised  HUD  format  was  not  included  as  part  of  the  formal  experiment  or  statistical  analy¬ 
sis,  however,  it  was  implemented  on  the  HUD  in  a  modified  form.  As  mentioned  above,  the  number 
labels  on  the  pitch  lines  did  not  rotate  with  rotation  of  the  pitch  lines,  but  remained  upright  regard¬ 
less  of  the  orientation  of  the  pitch  lines. 

Background  was  a  factor  in  this  experiment.  It  was  included  in  this  experiment  as  a  result  of 
discussions  with  the  working  group  on  situational  awareness  at  NATO.  The  background  against 
which  pilots  were  flying  was  considered  as  an  important  variable  in  loss  of  situational  awareness. 
Two  backgrounds  were  developed.  A  video  recording  of  two  sky  conditions  were  taped  during 
flight.  The  first  condition  was  a  recording  of  clear-blue  sky.  The  second  condition  was  a  recording 
of  a  star-lit  night  sky.  Those  video  tapes  were  displayed  on  the  22°  by  29°  field  of  view,  large 
screen,  virtual  image,  television.  This  television  system  was  developed  for  use  in  the  CREST  facility 
by  NADC  engineers. 

PROCEDURE 

The  subjects  were  tested  in  the  side-by-side  dynamic  cockpit  simulator  in  the  CREST  facility. 
The  subject  was  seated  in  the  right-side  seat  and  began  flying  the  simulator.  The  experimenter  was 
located  in  the  left-side  seat.  During  this  familiarization  phase  of  the  experiment  no  experimental  in¬ 
terventions  were  introduced.  The  subject  was  given  the  opportunity  to  experience  the  control- 
display  relationships  of  the  simulator,  the  flight  characteristics  involved  with  the  F/A-18  aerody¬ 
namic  model,  and  to  become  familiar  with  the  display  formats  used  for  the  experiment. 


The  subject  was  given  instructions  and  a  written  description  of  the  features  of  each  display 
which  would  be  used  for  attitude  assessment  Practice  trials  were  begun  when  the  subject  reported 
being  comforrable  with  flying  the  simulator  and  felt  able  to  control  the  simulation.  The  practice 
trials  were  included  to  assess  the  performance  of  each  subject  to  insure  that  the  criteria  of  90  per¬ 
cent  correct  responses  was  met  The  practice  trials  were  also  included  to  familiarize  the  subject 
with  the  experimental  procedure.  Thirty-six  practice  trials  were  completed,  six  trials  for  each  of 
three  formats  against  two  backgrounds. 

An  experimental  session  consisted  of  1 44  trials.  There  were  1 8  trials  for  each  possible  combination  of 
pitch  and  roll,  for  each  of  the  three  display  formats,  against  each  background.  The  subject  was  presented 
with  all  of  the  trials  for  each  display  for  one  background  before  continuing  on  to  another  background  or 
another  display.  The  order  of  presentation  of  the  display  formats  was  counter- balanced  across  the  experi¬ 
ment.  Only  the  ADI  ball  and  the  current  F/A-18  pitch-ladder  were  counter-balanced.  Since  the  revised  pitch- 
ladder  was  not  formally  included  as  part  of  the  experiment  or  statistical  analysis,  it  was  always  presented  last. 
The  background  condition  was  counter-balanced  for  each  display  format.  The  counter-balancing  was 
included  to  assure  against  any  carry-over  effects  which  might  confound  the  data  analysis. 
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An  experimental  trial  consisted  of:  ( 1 )  the  experimenter  selecting  the  appropriate  pitch  and 
roll  parameters,  (2)  alerting  the  subject  that  the  trial  was  to  begin,  and  (3)  the  experimenter  initia¬ 
ting  the  trial.  Once  the  trial  wss  initiated  the  subject  assumed  control  of  the  simulation.  The  sub¬ 
ject  controlled  the  simulator  for  a  brief  period,  maintaining  the  orientation  at  straight  and  level. 
Once  the  subject  felt  in  control  at  straight  and  level,  he  was  required  to  read  the  airspeed  and  enter 
it  into  the  center  keyboard  panel.  This  required  that  the  subject  look  away  from  the  display.  Upon 
pressing  the  "enter"  key  the  simulation  was  automatically  reoriented  to  the  pre-selected  pitch  and 
roll  orientation. 

The  subject's  task  at  this  point  was  to  use  the  control  stick  to  correct  the  simulation  back  to 
straight  and  level.  Once  the  subject  decided  that  he  had  the  simulation  back  to  straight  and  level 
and  had  it  under  control  he  was  required  to  squeeze  the  trigger  located  on  the  control  stick.  This 
stopped  the  simulation.  The  experimenter  then  entered  the  new  parameters  for  the  next  trial  and 
promptedithe  subject.  Rest  periods  were  taken  after  each  format  had  been  tested  under  both  back¬ 
ground  conditions. 

The  experimental  design  was  a  four-factor,  within-subjects,  repeated  measures  design.  The 
four  factors  were:  (1)  display  format,  (2)  background  condition,  (3)  pitch,  and  (4)  roll.  Each  sub¬ 
ject  received  every  possible  combination  of  treatments.  Data  analysis  was  completed  using  an  APL 
computer  language  program  (Gilman  and  Rose,  1974). 

DEPENDENT  MEASURES 

Two  dependent  measures  were  recorded:  (1)  decision  time  and  (2)  recovery  time.  Decision 
time  was  defined  for  this  experimant  as  the  amount  of  time  that  elapsed  between  pressing  the 
"enter"  key  on  the  center  panel  and  the  initiation  of  a  control  stick  movement.  Recovery  time  was 
defined  as  the  amount  time  that  elapsed  between  initiation  of  the  a  control  stick  movement  and 
when  the  trigger  on  the  control  stick  was  depressed. 


Subjective  preference  data  were  also  collected.  During  the  debriefing  session,  after  the  experimental 
trials  had  been  completed,  the  subject  was  asked  to  state  which  display  he  thought  he  performed  the  best 
with,  which  display  he  liked  the  most,  and  which  display  he  thought  provided  the  best  indication  of  attitude. 
The  subjective  data  revealed  substantial  individual  differences.  No  display  format  was  consistently  named 
more  often  than  the  others  as  a  response  to  any  of  the  questions. 

RESULTS 

The  mean  decision  times  for  each  format,  background,  pitch,  and  roll  combination  are  pre¬ 
sented  in  Table  2.  Results  of  the  statistical  analysis  for  decision  time  indicated  significant  main  ef¬ 
fects  of  background  (F  -  21.80,  p  <0.01)  and  pitch  (F  =  9.07,  p  <  0.01).  The  mean  decision  time 
to  determine  how  to  recover  was  significantly  faster  with  the  clear-blue  sky  than  when  performing 
the  task  against  a  star-lit  night  sky. 

Tukey's  HSD  (honestly  significant  difference)  post-hoc  multiple  comparison  test  was  em¬ 
ployed  to  determine  which  pitch  conditions  were  significantly  different  from  each  other  (Kirk, 
1968).  This  test  revealed  that  the  0°  pitch  condition  was  not  significantly  different  than  either  the 
55°  or  the  -55°  pitch  condition.  However,  the  latter  two  pitch  conditions  were  significantly  differ¬ 
ent  from  each  other.  The  means  and  the  results  of  the  HSD  post-hoc  test  for  background  and  pitch 
are  presented  in  Table  3. 
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TABLE  2 

MEANS  FOR  DECISION  TIME 


F/A-18  HUD 

ADI  BALL 

PITCH/ 

ROLL 

DAYLIGHT 

NIGHTLIGHT 

DAYLIGHT 

NIGHTLIGHT 

0°/0° 

1.95 

1.65 

2.36 

2.34 

0°/60° 

1.95 

1.85 

1.79 

2.00 

07120° 

1.75 

2.02 

1.89 

2.15 

07180° 

2.11 

2.26 

1.76 

2.15 

07-120° 

1.95 

2.06 

1.91 

1.96 

07-60° 

1.75 

2.08 

1.99 

2.03 

5570° 

2.01 

1.97 

1.63 

2.10 

55760° 

2.01 

2.09 

2.14 

2.40 

557120° 

1.67 

1.95 

1.99 

2.31 

557180° 

1.79 

1.92 

1.92 

2.31 

55° /-1 20° 

2.02 

2.05 

1.99 

2.30 

557-60° 

1.90 

2.12 

2.21 

2.49 

-55° /0° 

1.74 

1.85 

1.66 

1.92 

-55760° 

1.83 

1.86 

1.62 

1.94 

-557120° 

1.75 

1.92 

1.79 

1.93 

-55°/ 180° 

1.72 

1.97 

1.85 

1.93 

55°/- 120° 

1.88 

2.05 

1.78 

1.90 

-557-60° 

1.87 

1.96 

1.68 

1.75 
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TABLE  3 


RESULTS  OF  TUKEV'S  HSD  POST-HOC  MEAN  COMPARISON  TEST  FOR  DECISION  TIME 


EXPERIMENTAL 

CONDITION 


■  BACKGROUND 


DAYLIGHT 

1.878 


EXPERIMENTAL 

CONDITION 


=  PITCH 


NIGHTLIGHT 

2.043 


-56°  0°  55° 

1.840  1.976  2.054 


(Those  means  underlined  by  a  common  line  are  not  significantly  different  from  each 
other.) 
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A  significant  interaction  effect  was  also  revealed  by  the  overall JF  test.  The  interaction  of  dis¬ 
play  format  by  pitch  by  roll  was  significant  (£.**  3.68,  p  <  0.01).  Figure  4  presents  a  graph  of  this 
interaction.  The  interaction  effect  was  not  subjected  to  any  formal  post-hoc  statistical  analysis. 
Figure  4  does,  however,  provide  a  graphical  presentation  of  this  interaction.n 

The  mean  recovery  time  for  each  format,  background,  pitch,  and  roll  combination  are  pre¬ 
sented  in  Table  4.  The  overall_F  test  for  the  recovery  time  dependent  measure  revealed  significant 
main  effects  of  display  format  (JF  =  13.38,  p  <0.01),  pitch  (£"  30.36,  p  <0.01),  and  roll  (£  » 
19.49,  p  <0.01).  Tukey's  HSD  post-hoc  multiple  comparison  test  was  employed  to  determine 
which  pitch  and  roll  effects  were  significantly  different  from  the  others.  The  means  and  results  of 
the  HSD  test  for  display  format,  pitch  and  roll  are  presented  in  Table  5. 

The  HSD  test  for  recovery  times  revealed  that  the  0°  pitch  condition  was  significantly  differ¬ 
ent  than  either  the  55°  or  the  -55°  pitch  conditions.  The  latter  two  pitch  conditions  were  not  sig¬ 
nificantly  different.  The  HSD  test  also  revealed  that  the  0°  roll  condition  was  significantly  different 
than  all  of  the  other  roll  conditions.  It  also  revealed  that  the  60°  roll  condition  was  significantly 
different  from  the  180°  roll  condition.  No  other  roll  conditions  were  found  to  be  significantly  dif¬ 
ferent 

The  overall^  test  also  revealed  a  significant  interaction  of  pitch  and  roll  (F  =  7.38,  p  <  0.01). 
No  post-hoc  statistical  analysis  was  completed  for  the  interaction.  The  interaction  is  graphically 
presented  in  Figure  5. 
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Figure  4.  Display  Format  By  Pitch  By  Roll  Interaction 
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TABLE  4 

MEANS  FOR  RECOVERY  TIME 


PITCH/ 

ROLL 

F/A-18  HUD 

ADI  BALL 

DAYLIGHT 

NIGHTLIGHT 

DAYLIGHT 

NIGHTLIGHT 

0°/0° 

0.81 

0.87 

0.25 

0.18 

0760° 

11.35 

6.20 

6.75 

5.89 

07120° 

9.25 

12.20 

8.40 

7.15 

07180° 

9.36 

14.02 

8.43 

10.22 

07-120° 

11.74 

10.88 

8.91 

7.63 

0°/-60° 

9.04 

13.00 

6.60 

7.11 

5570° 

16.06 

15.09 

7.60 

9.07 

55760° 

21.35 

22.10 

11.75 

12.19 

55°/ 120° 

20.11 

18.63 

12.25 

13.65 

557180° 

21.22 

21.97 

12.23 

15.66 

557-120° 

19.44 

21.29 

13.83 

14.89 

557-60° 

18.33 

19.55 

13.65 

12.80 

-5570° 

9.82 

10.61 

9.97 

9.05 

-55760° 

16.34 

14.54 

11.35 

12.75 

-55°/ 120° 

15.76 

15.72 

13.72 

11.63 

-55°/ 180° 

17.47 

12.36 

15.65 

17.31 

55°/- 120° 

14.26 

19.90 

17.26 

12.95 

-557-60° 

14.33 

17.34 

13.18 

14.10 
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TABLE  5 

RESULTS  OF  TU  KEY'S  HSD  POST-HOC  MEAN  COMPARISON  TEST  FOR  RECOVERY  TIME 


EXPERIMENTAL 

CONDITION 


DISPLAY  FORMAT 


F/A-18  HUD 


14.552 


EXPERIMENTAL 

CONDITION 


=  PITCH 


01  -55° 

7.792  14.057 


ADI  BALL 
12.978 


55° 

16.063 


EXPERIMENTAL  nn,  . 
CONDITION  nULU 


01  60^  J2(£  -60°  -120°  180° 

7.447  12.789  13.206  13.251  14.484  14.646 


(Those  means  underlined  by  a  common  line  are  not  significantly  different  from  each 
other.) 
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Figure  5.  Pitch  By  Rolll  nteraction 
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DISCUSSION 

The  results  of  experiment  one  supported  the  hypothesis  that  the  ADI  ball  would  provide  a 
better  indication  of  attitude  (i.e.,  use  of  the  ADI  ball  m**i  *  'n  significantly  faster  decision  times). 
This  hypothesis  was  not  supported  by  the  results  of  ex  '-r  •  nt  two.  However,  the  results  from 
experiment  two  did*  indicate  that  the  ADI  bail  was  sone.  i:»  the.pjtch-laddgr  In  aiding  recovery 
froraAinusual  attitudes. 


The  second  hypothesis  of  experiment  one  was  not  supported.  The  revised  pitch-ladder  did  not  suipass 
the  current  pitch-ladder  in  terms  of  faster  decision  times.  An  informal  examination  of  the  revised  fonnnt  in 
experiment  two  also  indicated  that  the  revised  pitch-ladder  format  did  not  provide  a  better  indication  of 
attitude. 

The  significant  effects  from  experiment  two  of  pitch  and  roll  were  not  surprising.  For  both 
pitch  and  roll  the  0°  condition  was  significantly  different  from  the  other  conditions  for  recovery 
times.  It  is  obvious  that  it  would  require  more  time  to  recover  from  a  pitched  or  rolled  condition 
than  it  would  from  a  no-pitch  ora  no-roll  condition. 

The  experimental  support  of  the  hypothesis  that  the  ADI  ball  would  provide  a  better  indica¬ 
tion  of  attitude  than  a  pitch-ladder  is  consistent  with  other  research  in  this  area  (Roscoe,  Corl,  and 
Jensen,’ 1981).'  The  ADI  ball  is  more  consistent  with  the  generally  accepted  display  principle  of  pic¬ 
torial  realism.  The  ADI  ball  represents  a  contact  analog  more  so  than  a  pitch-ladder.  A  pitch-ladder 
is  more  symbolic  in  nature  than  a  ADI  ball  and,  therefore,  requires  more  cognitive  processing  to 
interpret. 

The  lack  of  experimental  suppoil  for  the  hypothesis  that  the  revised  pitch-ladder  would  be  better  than 
the  current  pitch-ladder  is  somewhat  surprising  in  view  of  the  resulls  reported  by  Taylor  ( 198d).  the  revisions 
to  the  current  pitch-ladder  were  based  on  results  obtained  by  Taylor  which  indicated  that  the  changes  he 
made  resulted  in  quicker  reaction  times  than  when  Ihe  changes  weie  not  included.  1  lie  discussion  by  Inyloi 
was  presented  in  terms  of  cognitive  processes  involved  in  assessing  altitude.  The  availability  of  redundant 
cues  which  could  be  processed  globally  should  aid  in  assessing  altitude. 


It  is  possible  that  the  changes  made  to  the  pitch-ladder,  although  thought  to  be  global,  may  not  have 
been  cognitively  processed  in  a  global  manner.  Several  of  the  modifications  were  apparently  unnoliceable. 

I  lowever,  Ihe  instructions  lo  the  subject  included  a  discussion  of  Ihe  lealmes  of  each  display  loimnl  which 
would  aid  him  in  assessing  attitude.  The  most  likely  explanation  of  tire  resulls  is  that  Hip  subject  tended  lo 
ignore  Ihe  more  subtle  cues  and  attended,  instead,  to  the  more  powerful  cues,  such  as  Ihe  orientation  of  the 
pilch-ladder  about  the  velocity  vector. 


RECOMMENDATIONS 

A  recommendations,  based  on  the  results  of  these  two  experiments,  would  be  to  include  an 
electromechanical  ADI  ball,  in  an  optimal  location,  within  the  display  suite  and  have  it  serve  as  the 
primary  instrument  for  attitude.  If  a  pitch-ladder  presentee  on  a  HUD  is  the  primary  flight  instru¬ 
ment,  as  it  is  in  the  F/A-18,  an  ADI  ball  should  be  present  as  a  secondary  instrument  in  a  location 
such  that  it  would  require  very  little  eye  translation  to  cross  reference  between  the  two  displays. 

Research  is  currently  proposed  which  would  examine  possibilities  other  than  retro  fitting  an 
ADI  ball  into  an  optimal  location.  Col.  Grant  McNaughton  from  the  U.S.  Air  Force  has  proposed 
several  pitch-ladder  formats  which  merit  consideration  (McNaughton,  1985).  They  provide  obvious 
cues  which  ieave  very  little  doubt  as  to  whether  they  would  be  processed  globally  or  not. 
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The  Command  Flight  Path  Display  (CFPD),  originally  developed  at  NADC  for  use  in  the  F- 14  may 
provide  another  possible  solution.  The  CFPD  is  an  integration  of  several  pieces  ol  essential  infoimnlion. 
With  specific  modifications  (e  g.,  frequency  separation  of  the  velocity  vector  In  leiation  to  the  flight  path)  it 
may  well  provide  an  excellent  indication  of  altitude.  The  CFPD  lias  been  flight  tested  will?  good  results  in  an 
F- 14  (Hoover,  Cronauer,  and  Shelley,  1985).  It  may  well  prove  to  be  an  excellent  solution  because  of  the 
integration  of  several  flight  parameters  into  one  readable  display. 


Another  possible  solution  may  result  from  an  advancement  in  display  technology.  A  new  up¬ 
front  controller  (UFC)  for  the  F/A-18  HUD  Is  currently  undergoing  a  technical  review  at  NADC. 
Included  in  this  new  UFC  is  a  three  inch  by  three  inch  flat  panel  display.  It  is  possible  that  an 
electronically-generated  ADI  "ball"  could  Ire  presented  on  this  display  surface.  If  this  electronically 
generated  ADI  "ball"  maintains  the  same  characteristics  of  an  electromechanical  ADI  ball,  it  may 
provide  a  good  secondary  attitude  indicator  which  would  be  optimally  located  so  that  very  little  eye 
translation  would  be  involved  in  cross-referencing. 

The  latter  proposal  is  currently  being  developed  as  a  potential  solution  addressing  the  problem 
of  loss  of  situational  awareness  in  the  F/A-18.  Continued  efforts  into  the  causes  and  contingencies 
of  loss  of  situational  awareness  will  lead  to  an  eventual  solution  to  this  problem. 
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